1. Field of the Invention
The present invention generally relates to an apparatus and a method of tracking an image feature, and more particularly, to an apparatus and a method of tracking an image feature using a block matching algorithm.
An image tracking system has various uses. For example, in a trespasser watching system in a building, a traffic observing system, a satellite tracking system, and a science observing system observing behavior of an animal for a long period of time, an image-feature tracking technique tracking movement of a target in the image moment by moment is required.
Further, to anticipate control action of an automobile or a mobile robot, the image-feature tracking technique for detecting and tracking a moving target, such as a pedestrian, is required. A visual measurement technique is also one of the expected image-feature tracking techniques.
In the visual measurement technique, a plurality of targets (in the case of an airplane, feature points such as top end points of left and right wings, a front end point, a tail end point) are set in one object. Then, by using tracking results of movement of each target, a three-dimension action or movement map of the object is calculated.
2. Description of the Related Art
In a prior-art tracking technique of an image of an object, a dedicated discriminative mark and a dedicated color plate were provided with the object. By using the mark and plate, a fast tracking processing may be provided by a simple image processing technique. However, in these cases, since the mark and the plate must be mounted to the object, the range of applying such image processing technique was limited.
On the contrary, recently, an image-feature tracking apparatus using a block matching algorithm is in practical use. FIG. 1 shows an illustration for explaining the prior-art image-feature tracking apparatus using the block matching algorithm. In FIG. 1, a frame f indicates an image frame at one instant of time, and a frame g indicates an image frame at the next instant of time.
In the prior-art image-feature tracking apparatus, first, in the frame f, an image block of a tracking target is defined as a reference block R. Coordinates of each picture element in the reference block R is represented by (x=1 to m, y=1 to n). The reference block R is generally referred to as a template. Next, in the frame g, the same sized image block is defined as a candidate block C. Coordinates of each picture element in the candidate block C are represented by (u+x, v+y), where u and v are respectively relative coordinates from a position of the reference block R. In the above situation, a matching D(u, v) of the reference block R and the candidate block C is calculated as a position of the candidate block C is varied within an area (-p.ltoreq.u.ltoreq.+q, -p.ltoreq.v.ltoreq.+q) of a search block S shown in FIG. 1.
In the above calculating process, from the position of the candidate block C giving a maximum value in the matching D, the amount of motion of the target between the frame f and the frame g may be calculated. Repeating the above-discussed process, a tracking process may be carried out.
In the process, an image, in which the reference block R is defined, is referred to as a reference image, and an image, in which the candidate block C is defined, is referred to as a search image. Since the image-feature tracking apparatus using the block matching algorithm requires no dedicated marks, such tracking apparatus is applicable for a variety of uses.
The block matching algorithm is generally given in the following equation. ##EQU1## where C(u+x, v+y), R(x, y) respectively indicate intensity of each picture element in the each block. In the above algorithm, an integral summation of an absolute value of a difference between the two image blocks is calculated. As a value of the integral summation is smaller, the value of the matching D increases. Therefore, the coordinates (u, v) when the value of the matching D(u, v) is smallest indicates the amount of motion of the target.
As a tracking mode, a fixed-template mode and a sequential-template mode have been developed. The fixed-template mode is used for tracking a specified pattern. The sequential-template mode is used for observing a more random motion such as movement of steam and watching a trespasser. The prior-art image-feature tracking apparatus using the block matching algorithm commonly uses the fixed-template mode.
In the fixed-template mode, first, the reference image (template image) of the object is set and stored, and the matching of the reference image with the sequentially-provided search image is calculated. Namely, the reference image is given at a specified time f0, and the candidate block C is successively updated at times f, f+1, f+2, . . . .
At these times, matching of the reference image at the time f0 and the candidate block C at the time f, matching of the reference image at the time f0 and the candidate block C at the time f+1, and matching of the reference image at the time f0 and the candidate block C at the time f+2, . . . are successively calculated. In the successive calculation, since the matching algorithm is carried out for one template, the specified pattern in the template may be tracked without missing any movement.
On the other hand, in the sequential-template mode, an instant motion vector of the image block of the object is measured. In this mode, without setting the specified template, matching the algorithm between two serial frames is repeated. Namely, matching of the image at the time f and the candidate block C at the time f+1, matching of the image at the time f+1 and the candidate block C at the time f+2, and matching of the image at the time f+2 and the candidate block C at the time f+3, . . . are successively calculated.
Applying this mode, a moving object may be recognized, and from many motion vectors (which are also referred to as optical flow), three-dimensional motion parameters may be calculated.
However, the above-discussed image-feature tracking apparatus using the block matching algorithm has the following disadvantages.
In the prior-art image-feature tracking apparatus using the block matching algorithm, when tracking a specified object, there is a problem in that an unstable tracking operation occurs when the object rotates or a distance between a camera and the object varies.